Monitoring and Evaluation Plan Primer for DRL Grantees 

I. What is a monitoring and evaluation plan? 

A monitoring and evaluation plan (M&E plan), sometimes also referred to as a 
performance monitoring or performance management plan, is a systematic and objective 
approach or process for monitoring project performance toward its objectives over time. 
The plan consists of indicators with baselines and targets; means for tracking critical 
assumptions; plans for managing the data collection process; and regular collection of 
data. Evaluations should be scheduled and carried out throughout the course of the 
program. 

Development of an M&E plan is integral to the planning of a program design. In fact, a 
rigorous M&E plan can be an effective tool for formulating a coherent and well-designed 
program proposal, both in revealing assumptions and exposing gaps in program planning. 
As the implementer modifies its program design, it is important to incorporate those 
changes into the M&E plan and vice versa. 

All program proposals being considered for DRL funding must include a comprehensive 
M&E plan. This plan is one of the key criteria in determining the competitiveness of a 
program proposal. 

II. Why is it important? 

As a planning document, an M&E plan forces an implementer to think systematically 
through each of the program components. During program execution, a well-designed 
plan helps the implementer to keep track of the program's progress and make adjustments 
if necessary. It is also a valuable tool for demonstrating the effectiveness and impact of 
a program, generating credible and useful information for both the implementer and 
donor that contributes to learning, improved performance, and accountability. For the 
donor, if a program has proven results, the M&E plan is a useful method for showcasing 
the program's success and documenting the implementer's track record. If the program is 
less than successful, the M&E plan can help to identify specific weaknesses. In either 
case, the M&E plan is useful for informing the donor's decisions about future budget 
allocations and programming. 

When DRL's Management Team conducts its bi-annual review of each grant, DRL uses 
the M&E plan as one way to compare the actual progress and impact of a program 
against the proposed activities and objectives. DRL compiles this information to measure 
the collective results of DRL programs and determine trends in program areas. 
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III. Goals of a monitoring and evaluation plan 

A M&E plan focuses on the performance of a project or program and examines its 
implementation plan, inputs, outputs and outcomes/results. A project is defined as an 
individually planned undertaking designed to achieve specific objectives within a given 
budget and time frame. The M&E plan should address the following questions: Did the 
project take off as planned? What problems and challenges, if any, did it face? Is it 
being effectively managed? Is it providing planned activities and other outputs in a 
timely fashion? If not, why? Will the project be able to meet its targets? What are its 
intermediary effects and impacts? What can be done to improve its performance and 
impacts? 

Most of the information for monitoring and evaluation can be gathered through reviews 
of project and program documents; developing and conducting surveys, focus group 
discussions and meetings with participants and other beneficiaries; and interviews with 
project staff, host country officials and other stakeholders. 

A solid M&E plan generally includes a mid-term and final evaluation. The mid-term 
evaluation can measure the interim progress of a program and identify areas that could be 
modified to improve performance and impact. The final evaluation can not only assesses 
the program's overall results, but also provide an analysis on lessons learned and make 
future recommendations. 

IV. Using a logic model to develop a good M&E plan (and program design) 

A recommended method for program design and development of an M&E plan is to use a 
logic model. A logic model is a depiction of the processes and targeted outcomes of the 
program. It should help the organization to specify strategic objectives, identify what 
resources are needed, develop indicators to measure progress and proposed results, and 
communicate the program's potential value. 

The components of a basic logic model include: 

■ Needs : The community need or problem as identified by the organization. 

■ Inputs : Human and financial resources used for the program implementation. 

■ Activities : Actions taken or work performed through which inputs are mobilized to 
produce outputs. 

■ Performance Indicators : Qualitative and quantitative measures or variables to be 
applied to the program activities. Performance indicators are directly linked to 
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measuring progress toward program objectives and are often a combination of 
monitoring and evaluation. 

■ Pathways : Linkages that specify how activities of a program lead to the expected 
outputs, outcomes, and impact of a program through each step of the logic model. 

■ Expected Outputs : Direct and measurable results expected from program activities. 
They should be tangible, visible and measurable products of program work. If they 
are sustainable beyond the activity, they may turn into program outcomes. 

■ Expected Outcomes : The short-term and medium-term effects of a program's outputs. 
Outcomes should reflect the results of program's activities and their near-term effects 
on program goals. 

■ Expected Impact : The long-term effects of a program, which is linked closely to the 
overall program objective. Such an objective could be as ambitious as reducing 
human rights violations in an authoritarian society, or it could be less ambitious, 
though equally important, such as adding greater female representation to a country's 
political parties. 

■ Assumptions : Hypotheses about factors or risks which could affect the progress or 
success of a program intervention. Our underlying beliefs about the program, the 
stakeholders or beneficiaries. 

■ External Factors : Factors which are not explicitly in the control of the program but 
which can have an important effect on the intended outcomes and impact, such as 
government policies or changes in the political situation in the country. 

While the logic model is represented in a linear trajectory, the relationship between 
factors may not always be unidirectional. Sometimes factors can mutually affect each 
other. Thus, the logic model should be viewed as a dynamic and evolving process, which 
should be re-evaluated and adjusted when conditions change. 

On the following page is a template of a logic model and its key components. It includes 
examples under each component. 
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Logic Model Template 



L^k Outcomes 



T 



Needs 



Inputs 



Activities 




Outputs 



Impact 



Needs of the 
community 
based on the 
social, 

political, and 
economic 
conditions of 
the country or 
region. 



Staff 


Conduct workshops, 


Time 


meetings 


Money 


Deliver services 


Materials 


Provide training 


Equipment 


Facilitate 


Partners 


Partner 




Products created 

Resources developed 

Serviced delivered 

Participants (e.g., 
NGOs, local citizens, 
media, host country 
government officials) 
reached 

Measured with 
Performance indicators 



Increased awareness, 
knowledge, or attitudes 

Improved skills 

Change in behavior, 
practice, or 
decisionmaking 

Policy change 

Measured with 
Performance indicators 



Change in social, 
economic, or civic 
condition 




Assumptions 



External Factors 



Country's political and economic condition 
Skills and knowledge level of intended beneficiary 
Implementer's organizational capacity 
Level of engagement by stakeholders 



Change in government policies, such as NGO restrictions 
Change in political situation, such as a coup or civil unrest 
Involvement from other donors and implementers 
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V. The difference between a goal and an objective 

The terms goals and objectives are often used interchangeably, but there are distinct 
differences between the two, mainly in terms of their level of concreteness. Both terms 
imply the target desired as a result of one's efforts. However, goals are less structured, 
whereas objectives are concrete and viewed as targets under the general goal. Also, 
because goals tend to be lofty in nature, it is not possible to measure the accomplishment 
of one's goals. But one goes about measuring progress towards a goal by setting tangible 
objectives with clear timelines, which are possible to measure. For example, country X 
has set the goal of establishing a free and independent media representing its diverse 
populations. One program strategic objective is to train a cadre of female journalists on 
investigative journalism from researching to reporting, while another program strategic 
objective is to build the management skills of the station managers of a new independent 
radio station so that they can effectively run the station. Achievement of these objectives 
suggests advancement towards the end goal of actualizing an independent media. 

Below is a table delineating the key differences between goals and objectives. 



Goals 


Objectives 


Broad and visionary in nature 


Narrow and time bound 


General intentions 


Precise 


Not resource dependent 


Resource dependent 


Intangible 


Tangible 


Abstract 


Concrete 


Can't be validated 


Can be validated 



DRL details the goals of its funding priorities in the Request for Proposals. Potential 
grantees should identify the strategic objectives for its programs in its proposal and 
briefly explain how their program strategic ob jectives fit within DRL 's goals. 

VI. Developing valid and reliable performance indicators 

Performance indicators are used to observe progress and measure actual results compared 
with expected results. Performance indicators should be linked to the program's strategic 
objectives and focus on outputs and outcomes, as identified by the program's logic 
model. In differentiating between outputs and outcomes, it can be useful to think of 
outcomes as developmentally significant changes that impact the beneficiaries, while 
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outputs are lower-level steps that are not developmentally significant but are essential in 
achieving results. Outputs may impact the beneficiaries but in a more limited way than 
outcomes. The link between outputs and outcomes should always be direct. For 
example, the activity could be training community journalists on investigative journalism 
through a series of workshops. One output could be the number of community journalists 
who are trained. The outcome could be the new skills that participants' develop as a 
result of the workshops. However, it may take many outputs from several activities over 
time to achieve an outcome or, ultimately, measureable impact. For example, the 
program objective could be developing quality community journalists in the field of 
investigative journalism. Besides participating in the series of workshops, leading 
investigative journalists mentor these community journalists in this trade over the course 
of a year. The combination of the workshops and mentorship is meant to elevate the 
quality of work produced by these community journalists. 

Measures should be in quantifiable terms and clearly defined. Qualitative indicators, 
though more rare than quantitative indicators, are acceptable if they provide a reliable 
means to measure a particular phenomenon or attribute. A good guide for determining 
performance measures is the acronym SMART. They should be: 

■ Specific 

■ Measurable 

■ Attainable 

■ Realistic 

■ Timely 

Back to the example described above, one performance indicator is the quality of the 
participants' written work. This can be measured by having journalism experts 
systematically compare the participants' work written before and after their participation 
in the workshops and the mentorship program. 

Besides defining the performance indicators in the M&E plan, it is also important to set 
baselines and targets for each indicator and describe the methodology for measuring the 
performance indicators (e.g., pre- and post-test surveys, focus groups, interviews, etc.), 
including the data source (e.g., training sign- in sheets, website hits, survey data) and the 
frequency of measurement (e.g., after each workshop, quarterly, annually). 

To assess trends across the four program areas under the Foreign Assistance Framework 
(Human Rights and Rule of Law, Civil Society, Political Process and Competition- 
Building, and Good Governance), DRL has identified a set of standard output-oriented 
indicators within each program area. In addition to program-specific indicators, grantees 
are also required to report on any of these indicators that are relevant to their program 
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in the quarterly progress reports, by each quarter and cumulative over the life of the 
grant, so that DRL can meet its own Department of State reporting requirements. 

VII. Use of a Third-party or External Evaluator 

Ideally, an experienced evaluator should be involved with the program from the very 
beginning. The evaluator should collect data relevant to the program's activities and 
evaluate the implementer's impact. One of the better ways of ensuring that the 
monitoring and evaluation is done thoroughly and effectively is to hire a third-party or 
external evaluator - someone who is not personally involved with the project and, 
preferably, not directly related to the organization. An external evaluation is not a 
requirement for DRL programs, but proposals, which include one, tend to be rated more 
competitively by panelists. An external evaluator can take a fresh look at the program 
and can be more objective when collecting and analyzing data and presenting the results. 
Resources for finding a program evaluator include colleges and universities, research 
firms, and nonprofits or other specialized organizations either in the U.S. or in the 
country where the program is being implemented. 

If the program includes an independent evaluation, DRL should be consulted on the 
hiring of the independent evaluator and evaluation methodology, where applicable. DRL 
should also receive the independent evaluator 's reports. 

VIII. Key definitions 

Activity: A specific action or process undertaken over a specific period of time by an 
organization to convert resources to products or services to achieve results. Related 
terms: Program and Project. 

Assumptions: Hypotheses about factors or risks, such as underlying beliefs about the 
program, the stakeholders or beneficiaries, which could affect the progress or success of a 
program. 

Baseline: Information collected before or at the start of a project or program that 
provides a basis for planning and/or assessing subsequent progress and impact. Ideally, 
the baseline should be measured just before the implementation of activities that 
contribute to achieving the result. 

Benchmark: A standard against which results are measured. Related terms: Milestone, 
Target. 

Beneficiaries: The individuals, groups, or organizations that benefit from an activity, 
project, or program. 
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Data: Information collected by a researcher or program implementer. Data gathered 
during an evaluation are analyzed to yield findings that serve as the basis for conclusions 
and recommendations. 

Data Collection Methods: Techniques used to identify information sources, collect 
information, and minimize bias during an evaluation. Examples include surveys, focus 
groups, and interviews. 

Evaluation: A systematic and objective assessment of an on-going or completed project 
or program. Evaluations are undertaken to (a) improve the performance of existing 
programs, (b) assess their effects and impacts, and (c) inform decisions about future 
programming. Evaluations are formal analytical endeavors involving systematic 
collection and analysis of qualitative and quantitative information. 

Evaluation Design: The methodology selected for collecting and analyzing data in order 
to reach defendable conclusions about a program or project efficiency and effectiveness. 

External Evaluation: The evaluation of a project or program conducted by entities 
and/or individuals not directly related to the implementing organization. Related term: 
Independent Evaluation. 

External Factors: Factors which are not explicitly in the control of the program but 
which can have an important effect on the intended outcomes and impact, such as 
government policies or changes in the political situation in the country (e.g., a law 
restricting NGO activities, coup). 

Goal: The higher-order objective to which a project or program is intended to contribute. 
A goal should be lofty in nature and not resource dependent. 

Impact: A result or effect that is caused by or attributable to a project or program. It can 
also be a significant and measurable change affecting project beneficiaries. Impact is 
often used to refer to higher level effects of a program that occur in the medium or long 
term, and can be intended or unintended and positive or negative. For example, if an 
education program trains teachers, the number of teachers trained and skills acquired by 
teachers are the outputs, the improvement in teacher's quality of teaching is the outcome, 
and the improvement in the education of those teachers' students is the impact. Related 
terms: Result, Outcome. 

Independent Evaluation: An evaluation carried out by entities and persons not directly 
involved in the design or implementation of a project or program. It is characterized by 
full access to information and by full autonomy in carrying out investigations and 
reporting findings. Related term: External Evaluation. 

Indicator (or Performance Indicator): A particular characteristic or dimension used to 
measure intended changes. Indicators are used to observe progress and measure actual 
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results compared with expected results. Indicators answer "how" or "whether" a project 
is progressing toward objectives. Indicators can be expressed quantitatively and should 
be objective and measureable (e.g., numeric value, percentages, indices). Indicators 
which are qualitative are less common, though acceptable if they provide a reliable means 
to measure a particular phenomenon or attribute. Examples of indicators include: number 
of gender-based violence survivors provided social support services and percent change 
in knowledge about investigative journalism from workshop participants. 

Input: Resources used to produce an output or results, such as technical assistance, 
commodities, capital, and training. Resources provided for program implementation. 
Examples are money, staff, time, facilities, equipment, etc. 

Internal Evaluation: Evaluation conducted by those who are implementing and/or 
managing the intervention or program. Related term: Self-Evaluation. 

Logic Model: A logic model, often a visual representation, provides a road map showing 
the sequence of related events connecting the need for a planned program with the 
programs' desired outcomes and results. It should identify strategic project elements 
(e.g., inputs, outputs, outcomes, impact) and their relationships, indicators, and the 
assumptions or risks that may influence success and failure. 

Mid-term Evaluation: Evaluation performed towards the midpoint of the program or 
project implementation. 

Milestone: Specific interim events, products, or steps in a process that convey progress 
toward completion of a deliverable or result. Milestones tend to be output-oriented. 
Examples include: 5 trainings held, working group established, and law drafted by the 
working group. Related terms: Benchmark, Target. 

Monitoring: The performance and analysis of routine measurements to detect a change 
in status. Monitoring is used to inform managers about the progress of an ongoing 
intervention or program, and to detect problems that may be able to be addressed through 
corrective actions. 

Objective: A statement of the condition or state one expects to achieve. An objective 
should be concrete, time-bound, and measureable and viewed as targets within the 
general goal. Related term: Strategic objective. 

Outcome: Specific changes in events, occurrences, or conditions, such as attitudes, 
behaviors, knowledge, skills, status, or level of functioning, that are caused by or 
attributable to outputs or program activities. These often are expressed at an individual 
level among program participants. For example, an education program might train 
teachers; the number of teacher trained and the number of skills acquired by the trained 
teachers are outputs. Training teachers would presumably lead to improved instruction, 
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which is an outcome. Outcome is often used to refer to more immediate and intended 
effects. Related term: Result. 

Output: A tangible, immediate, and intended product or consequence of an activity 
within an organization's manageable interest. Program deliverables are generally 
considered outputs. Examples include: number of journalist trained, number of media 
articles written, and number of manuals distributed. 

Program: A set of activities implemented by a defined set of implementers and designed 
to achieve specific objectives over a specified period of time that may cut across sectors, 
themes and/or geographic areas. Related terms: Activity, Project. 

Program Evaluation: Evaluation of a set of activities designed to attain specific global, 
regional, country, or sector development objectives. A program is a time-bound 
intervention involving multiple activities that may cut across sectors, themes and/or 
geographic areas. 

Project: An individually planned undertaking designed to achieve specific objectives 
within a given budget and time frame. Related terms: Activity, Program. 

Result: A significant, intended (or unintended), and measureable change in the condition 
of a beneficiary or a change in the host country, institutions, or other entities that affect 
the customer/beneficiary directly or indirectly. Related term: Outcome. 

Scope of Work: A written description of the objectives, tasks, methods, deliverables and 
schedules for an evaluation. 

Self-Evaluation: An evaluation by those who are entrusted with the design and 
implementation of a project or program. Related term: Internal Evaluation. 

Strategic Objective: The most ambitious result that the program can affect and for 
which it is willing to be held accountable within the planned time period. Related term: 
Objective. 

Target: An expected value or level of an indicator at a specified point in time in the 
future. The target shows the expected level of achievement or progress in achieving the 
associated result and forms the standard against which actual results are compared and 
assessed. A target is defined for each indicator as part of the M&E plan. Related terms: 
Benchmark, Milestone. 
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IX. Useful resources 



American Evaluation Association 
www.eval.org/ 

United Way, Outcome Measurement Resource Network 
http://www.liveunited.org/Outcomes/Resources/MPO/excerpts.cfm 

U.S. Foreign Assistance website 
w w w . foreignas sistance . net 

World Bank, Independent Evaluation Group 
http://www.worldbank.org/oed/ 

World Bank, Independent Evaluation: Principles, Guidelines and Good Practice 
http://siteresources.worldbank.org/INTDGF/Resources/Evaluation&LearningNote.pdf 



X. Sources 

Logic Model Development Guide: 
http://www.exinfm.com/training/pdfiles/logicModel.pdf 

U.S. Foreign Assistance Evaluation Glossary: 
http ://communities . usaidallnet . go v/fa/node/ 1 498 

U.S. Foreign Assistance Evaluation Guidelines: 
http ://communities . usaidallnet . go v/fa/node/ 1 499 
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